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Abstract  -  Simultaneous  target  tracking  and  identification 
through  feature  association,  attribute  matching,  or  blob 
analysis  is  dependent  on  spatio-temporal  measurements. 
Improved  track  maintenance  should  be  achievable  by 
maintaining  coarse  sensor  resolutions  on  maneuvering 
targets  and  utilizing  finer  sensor  resolutions  to  resolve 
closely-spaced  targets.  There  are  inherent  optimal 
resolutions  for  sensors  and  restricted  altitudes  that 
constrain  operational  performance  that  a  sensor  manager 
must  optimize  for  both  wide-area  surveillance  and 
precision  tracking.  The  advent  of  better  optics, 
coordinated  sensor  management,  and  fusion  strategies 
provide  an  opportunity  to  enhance  simultaneous  tracking 
and  identification  algorithms.  We  investigate  utilizing 
electro-optical  (EO)  and  Infrared  (IR)  sensors  operating 
at  various  resolutions  to  optimize  target  tracking  and 
identification.  We  use  a  target-dense  maneuvering 
scenario  to  highlight  the  performance  gains  with  the 
Multiresolution  EO/IR  data  association  (MEIDA) 
algorithm  in  tracking  crossing  targets. 
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1  Introduction 

The  problem  of  multitarget  tracking  and  identification 
(ID)  is  a  subset  of  sensor  fusion,  which  includes  filtering, 
estimation,  and  prediction.  One  of  the  prominent  tracking 
algorithms  is  the  Joint  Probability  Data  Association  Filter 
(JPDAF)  [1].  This  algorithm  seeks  to  track  a  set  of  objects 
from  only  positional  information,  but  improvements  are 
underway  to  use  signal-detection  analysis  to  track  an 
object  based  on  the  highest  signal  return  [2].  One  way  to 
enhance  data  association  is  to  use  EO/IR  target  attributes 
to  mitigate  clutter  [3-7].  To  further  enhance  the 
capabilities  of  the  JPDAF  algorithm,  it  would  be  useful  for 
a  sensor  to  not  only  get  the  position  of  the  target,  but  also 
the  target  identity.  However,  a  true  identity  might  not  be 
known,  so  believable  measurements  must  be  used.  A 
combined  track  and  ID  algorithm  can  improve  track 
quality,  mitigate  clutter  confusion,  and  enhance  target  ID. 
Kinematic  and  ID  measurements  can  detect,  track,  and 
classify  targets  of  interest.  The  ultimate  objective  of  the 
tracker  includes  identifying  targets  as  they  move  through 
space.  In  a  dynamic  and  uncertain  environment,  the 
tracker  must  associate  the  correct  target  to  the  position 
measurements.  All  neighbors  data  association  algorithms 
[1,  8]  calculate  a  posteriori  probabilities  of  allocating  the 
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position  measurements  to  candidate  potential  targets. 
Likewise,  other  multisensor  multiplatform  fusion 
algorithms  identify  targets  from  multiple  look  sequences 
of  sensor  data  [9].  The  merging  of  these  algorithms  can  be 
accomplished  by  investigating  the  mathematics  of  the 
algorithms.  Track  fusion  uses  kinematic  measurements 
and  ID  fusion  uses  target- feature  measurements  to  update 
state  matrices.  We  seek  to  simultaneously  track  and 
identify  targets  by  utilizing  the  merits  of  data  association 
from  multilevel  data  associations.  The  Joint-Belief 
Probabilistic  Data  Association  (JBPDA)  algorithm  using 
Bayesian  and  Dempster- Shafer  updates  was  proposed 
[10].  Similar  methodologies  include  DSm  tracking  [11]. 

Target  tracking  and  ID  utilizes  both  kinematic  and  target 
features  to  track  targets.  Many  tracking  algorithms  either 
use  kinematic  information  or  track  features  to  separate 
closely  spaced  targets.  A  promising  strategy  is  to  use  the 
appropriate  sensor  resolution  for  the  target  movement 
context.  To  determine  the  coarse  information,  we  can 
utilize  sensors  at  low  resolutions  to  obtain  the  length-to- 
width  ratio  of  the  target.  High  resolution  sensors  capture 
the  feature  information  used  to  identify  the  target.  In  either 
coarse  or  fine  resolution,  three  models  are  necessary  (1) 
target  dynamics,  (2)  target  identification,  and  (3)  sensor. 
The  target  dynamics  include  such  models  as  constant 
velocity  or  constant  turn  (which  is  utilized  in  an 
Interactive  Multiple  Model  [IMM]  -  [1,  8,  12])  and  a 
move-stop-move  analysis.  Target  ID  models  assist  in  the 
measurement-to-track  association  (i.e.  EO/IR  sensors, 
shown  in  Figure  1).  Sensor  models  include  the  resolution 
and  target  movement.  All  three  can  update  the  pose 
estimate  of  the  target  for  predicting  the  next  location. 
Assuming  the  sensors  have  a  fixed  sampling  rate,  finer 
resolutions  would  help  resolve  interacting  targets  while 
decreasing  wide  area  surveillance.  Obviously,  there  are 
more  pixels  on  target  when  the  sensor  is  in  high 
resolution. 


Figure  1.  (a)  EO  image  and  (B)  IR  image  of  targets. 


One  example  is  having  an  EO/IR  platform  where 
analysis  has  been  applied  to  target  recognition  [13,14,15]. 
We  would  want  the  EO  to  capture  targets  in  good  lighting 
conditions  and  IR  to  capture  thermal  images  at  night.  An 
IR  imager  can  be  used  at  all  times  of  the  day,  albeit,  the 
performance  may  degrade  during  short  thermal  cross-over 
periods  in  the  morning  and  evening.  We  would  want  a  fine 
resolution  video  sensor  on  closely  spaced  targets  and  a 
coarse  resolution  on  separated  targets.  To  determine  the 
tradeoffs  of  track  filters,  we  can  use  a  Markov  weighting 
to  determine  which  sensor  mode  and  resolution,  which 
target  dynamic  and  speed  model,  and  what  fusion  strategy 
for  pose  updates. 

Many  tracking  algorithms  incorporate  ID  information, 
either  using  the  position  to  locate  and  detect  the  target  [7], 
use  kinematic  information  to  facilitate  target  recognition 
[17]  ,  and  an  intersection  of  kinematic  and  ID  information 
[10, 16].  Also,  there  are  multiresolution  algorithms  such  as 
using  wavelets  [9].  To  facilitate  a  pragmatic  use  of 
resolution  capabilities,  it  would  be  desirable  to  use  low 
resolution  for  wide  area  surveillance  and  high  resolution 
to  discern  closely  space  objects. 

In  data  association  tracking  approaches,  typical 
tracking  is  provided  through  position  measurements.  The 
problem  is  that  the  tracker  must  isolate  the  target  of 
interest  from  the  cluttered  position  measurements.  If 
position  measurement  information  is  dense,  the  tracker  can 
make  an  incorrect  assignment  of  the  position 
measurements  to  target  tracks.  As  an  example,  Figure  2 
below  shows  a  case  in  which  the  position  measurements 
cause  the  tracker  to  get  confused.  In  this  case,  object  l’s 
and  object  2’s  position  measurements  fall  within  the 
kinematic  gates  of  both  objects. 


Object  2  trajectory 


Figure  2.  Data  Association  Problem  with  only  Position 
Measurements. 

As  we  can  see  on  the  far  left  of  Figure  2,  a  kinematic 
gate  can  isolate  position  measurements  that  are  near  the 
predicted  measurement  for  each  object’s  track.  In  the 
case  that  one  of  the  true  measurements  falls  within  the 
kinematic  gate  of  the  predicted  position,  that  measurement 
would  be  designated  as  the  true  position  measurement. 
The  position  measurement  of  object  2  would  be  assigned 
to  object  l’s  track  if  position  measurements  from  another 
object  fell  within  the  predicted  kinematic  gate  of  the 


object  track  as  shown  in  the  middle  of  Figure  2,  which 
would  be  an  incorrect  assignment.  Once  the  tracker  locks 
on  to  another  object,  or  uses  the  position  hits  of  the  other 
object’s  clutter,  the  tracker  assumes  that  the  hits  of  the 
second  object  are  true  hits  for  the  first  object  as  shown  on 
the  far  right  of  Figure  2. 

One  way  to  correct  for  this  measurement-to-target 
assignment  mistake  is  to  leverage  other  information,  such 
as  the  target  identity  to  help  resolve  which  position 
measurements  are  assigned  to  specific  object  tracks.  For 
example,  we  could  use  a  high  resolution  EO/IR  sensor  to 
ID  a  moving  target  and  use  the  false  and  true  EO/IR  scans 
as  positional  information  [10].  To  illustrate  how  the  ID 
information  may  help  in  data  association,  Figure  3 
illustrates  the  process  of  how  a  target-ID  can  refine  the 
positional  measurement  to  select  the  validated 
measurement  from  the  cluttered  measurements. 


Figure  3.  Data  Association  using  ID  and  Position 
Measurements. 

A  few  tracking  and  ID  algorithms  have  been 
proposed  [10,11,16,17].  These  approaches  use  target  track 
information  to  cue  target  recognition  and  detection.  We 
seek  to  expand  on  this  idea  by  allowing  for  a 
multiresolution  capability  to  track  targets  in  a  wide-area 
search  mode  and  ID  relevant  targets  in  precision  tracking. 
Identification  goes  beyond  recognition  by  assigning  a 
single  target  ID  to  each  target.  In  this  paper,  we  are 
interested  in  controlling  the  sensor  resolution  to  aid  the 
track  and  ID  process  with  a  series  of  sensors. 

This  paper  develops  a  multiresolution  EO/IR  track 
and  identification  data  association  (MEIDA)  technique  to 
distinguish  between  multiple  moving  targets  in  clutter. 
Section  2  overviews  the  EO/IR  target  identification  fusion 
based  on  the  data  sets.  Section  3  discusses  track  and  ID 
data  association.  Section  4  describes  the  problem 
formulation  and  Section  5  details  the  mathematics  of  the 
algorithm.  Section  6  presents  results  and  Section  7  draws 
conclusions. 

2  EO/IR  Target  Recognition  &  Fusion 

In  the  case  of  day/night  analysis,  many  surveillance 
systems  incorporate  both  electro-optical  and  infrared  (IR) 
camera  systems  to  be  able  to  capture  the  visible  spectrum 
in  the  day  and  thermal  imaging  at  night.  EO/IR  images  are 


subject  to  the  camera-to-target  range. [18]  In  search  mode, 
it  would  be  desirable  to  stay  far  way  to  find  targets  (track 
initiation).  As  targets  are  acquired,  the  tracker  operates  in 
a  track  maintenance  mode  to  follow  targets;  however,  to 
identify  the  targets  when  they  are  closely  spaced,  the 
sensor  manager  must  change  the  resolution.  While  each 
sensor  is  better  for  a  given  lighting  condition,  the  sensor 
suite  simultaneously  outputs  both  images  (see  Figure  1), 
for  a  given  resolution. 

Synthetic  target  images  were  generated  for  360 
degrees  for  4  targets  at  three  resolutions  (high,  medium, 
and  low).  The  highest  resolution  data  was  generated  to 
yield  200  times  more  pixels  on  target  than  the  lowest 
resolution,  and  the  medium  resolution  had  10  times  more 
pixels  on  target  than  the  lowest  resolution.  Each  image 
can  facilitate  target  ID.  A  set  of  training  images  and  a  set 
of  test  images  were  created  for  target  ID.  The  ID  consisted 
of  determining  the  associated  confusion  matrix  for  the  set 
of  targets  at  each  resolution  for  each  of  the  360  degrees. 
The  resolution  is  a  function  of  the  range  and  the 
corresponding  number  of  pixels  on  target  is  shown  in 
Figure  4.  The  gaps  in  the  data  are  due  to  a  percentage  of 
random  samples  being  removed  from  the  test  data  to 
decrease  simulation  run  times.  Obviously,  target 
recognition  improves  with  more  pixels  on  target. 
Although  greater  sensor  resolution  is  the  biggest 
contributing  factor,  vehicles  orientation  also  contributes  to 
the  number  of  pixels  on  targets  as  can  be  scene  in  Figure 
4  at  broadside  and  head  and  tail  on  perspectives.  Thus,  the 
target  recognition  would  be  better  with  higher  resolution, 
but  is  still  subject  to  the  orientation  and  size  of  the  target. 


Sensor  Resolution  Over  Aspect 


Figure  4.  Pixels  on  target. 

The  target  recognition  is  better  at  the  high  resolution 
versus  the  low  resolution,  as  shown  in  Figure  5.  From 
Figure  5,  we  see  that  at  the  high  resolution,  the  target  ID  is 
of  higher  fidelity  than  that  of  the  low  resolution.  Since  the 
EO/IR  images  were  synthetically  generated,  the  ID  results 
were  a  function  of  operating  conditions  (i.e.  we  have 
better  ID  performance  with  fewer  operating  conditions  of 
atmospheric,  lighting,  etc).  The  belief  values  were 
calculated  using  a  Bayesian  likelihood  based  on  peak 


signal  returns  and  target  shape  features.  Tike  wise,  the 
fused  EO/IR  results  of  Figure  5  are  a  function  of  selecting 
the  optimum  match  of  either  the  EO  or  the  IR  image 
(which  we  will  use  in  the  simulation  -  Section  6).  Finally, 
a  discrete  selection  of  the  target  was  made  from  a  set  of  4 
targets  with  one  member  of  the  set  being  an  unknown 
quantity.  In  many  cases  for  the  low  resolution,  the  target 
selection  was  unknown  (many  points  at  the  origin  of  the 
polar  plot).  In  this  case  of  low  belief,  the  tracker  just  uses 
the  kinematic  measurement,  with  no  selection  of  the  target 
ID  to  weight  the  measurements. 

Low  Resolution  High  Resolution 


Figure  5.  Target  ID  at  low  and  high  resolutions. 

(Different  colors  represent  different  target  Pr(ID)  for 

various  aspects  versus  the  target  set). 

3  Track  and  ID  Fusion 

The  ability  to  perform  track  and  ID  fusion  requires  sensor- 
processed  classifications  from  different  levels.  These 
levels  could  be  generic(car),  feature( wheeled),  type(sedan) 
or  specific(license  plate).  Like  multitarget  data 
association  algorithms  for  accurately  tracking  targets  in 
the  presence  of  clutter,  we  assume  that  detected  targets 
can  be  tracked  from  a  sequence  of  center-of-gravity  and 
pose  positional  data.  However,  for  a  given  sensor/target 
scenario,  we  assume  detected  classifications  can 
effectively  discern  target  ID.  ID  information  can  be 
achieved  either  through  experience  of  target  movement, 
training,  or  predicted.  For  example,  identifying  a  target 
requires  the  correct  orientation  and  speed  estimate.  Two 
targets  of  the  same  type  may  be  crossing  in  space,  but 
since  they  can  not  occupy  the  same  location,  they  would 
each  have  a  different  orientation  relative  to  a  sensor.  By 
exploiting  the  orientation,  velocity,  and  multiresolution 
EO/IR  feature  information,  each  target  can  be  assessed  for 
the  correct  track-ID  association. 

The  capability  of  a  sensor  to  track  and  ID  targets 
simultaneously  includes  finding  the  target  center  for 
tracking,  determining  the  target  pose,  and  searching  the 
neighboring  characteristics  for  discerning  salient  features 
for  association  to  a  specific  target  type.  By  partitioning 
kinematic  and  ID  sensor  data,  associations  at  different 
levels  can  be  used  for  either  coarse(track)  or  fine  (ID) 
target  analysis.  For  example,  features  [10]  can  be  used  to 
get  an  ID  with  uncertainty;  however  if  many  features  are 
fused,  the  identity  improves  and  helps  eliminate  clutter. 


The  tracker  must  use  the  available  features  to  discern  an 
object  ( identify  a  target)  which  is  a  subset  of  Automatic 
Target  Recognition  (ATR).  Certain  features  are  inherently 
more  useful  in  recognizing  a  target  than  others.  For 
instance,  identifying  a  large  car  versus  a  small  car  would 
result  from  an  analysis  of  the  length-to-width  ratio. 
However,  obtaining  these  features  is  a  function  of  sensor 
resolution.  Additionally,  decoupling  information  can  be 
used  for  a  single-platform  observer  to  fuse  information 
from  a  sequence  of  sensor  data  or  for  a  multiple-platform 
scenario  [9]  in  which  fusing  is  performed  from  different 
geometrical  positions.  Further  information  on  the 
development  of  the  belief-ID  derivation  is  found  in  [10]. 

The  problem  of  track  level  and  ID-level  fusion  has 
characteristic  tradeoffs  about  which  the  tracker  must 
decide.  For  close  targets,  it  is  useful  to  keep  an  accurate 
track  on  multiple  targets.  The  intelligent  processor 
performs  target-to-ID  association  at  multiple  levels  and 
can  either  track  targets  at  a  low  resolution  or  ID  targets  at 
a  higher  resolution.  By  leveraging  knowledge  about  target 
types,  fusion  algorithms  can  significantly  reduce 
processing  time  for  tracking  and  identifying  targets.  For 
separated  targets,  resources  may  exist  to  identify  each 
target.  Hence,  due  to  a  limited  set  of  resources  and/or 
processor  time,  a  trade-off  exists  between  the 
identification  and  tracking  of  a  target. 

4  Problem  Formulation 

Consider  an  environment  in  which  a  tracker  is  monitoring 
multiple  moving  targets  with  stationary  clutter.  By 
assumption,  the  tracking  sensor  is  able  to  detect  target 
signatures.  Assume  that  the  2-D  region  is  composed  of  T 
targets  with / features.  Dynamic  target  measurements  z  are 
taken  at  time  steps  k ,  which  include  target  kinematic  and 
identification  features  z(k)  =  [x  t  (k),  f  b. . .  f  n]. 

Any  sensor  can  measure  independently  of  the  others, 
and  the  outcome  of  each  measurement  may  contain 
kinematic  or  feature  variables  indicating  any  target.  The 
features  for  each  sensor  are  similar,  but  need  to  be 
extracted  and  applied  to  the  separate  targets  for 
classification.  The  probability  density  of  each 
measurement  depends  on  whether  the  target  is  actually 
present  or  not.  Further  assume  that  a  fixed  number  of 
kinematic  and  feature  measurements  will  be  taken  at  each 
time  interval,  where  we  model  the  clutter  composing 
spurious  measurements.  A  final  decision  from  the  MEIDA 
algorithm  is  rendered  as  to  which  [x,  y ]  measurement  is 
associated  with  the  target-type. 

The  multisens  or -multitar get  tracking  and 
identification  problem  is  to  determine  which  measured 
kinematic  features  should  be  associated  with  which  ID 
features  in  order  to  optimize  the  probability  that  targets  are 
tracked  and  identified  correctly  after  z  measurements.  The 
multilevel  feature  fusion  problem  is  formulated  and  solved 
by  using  concepts  developed  using  the  belief  filter  [10]. 
For  the  symmetric-target  case,  the  "association  rule"  uses 
the  measurement  with  the  highest  target  probability. 


5  Track  and  ID  Data  Association 

5.1  Tracking  Belief  Filter 

The  target  state  and  true  measurement  are  assumed  to 
evolve  in  time  according  to: 

x(k+  1)  =  F(&)  x(&)  +  v(&)  (1) 

z(k)  =  U(k)  x(k)  +  w(k)  (2) 

where  v(&)  and  w(&)  are  zero-mean  mutually  independent 
white  Gaussian  noise  sequences  with  known  covariance 
matrices  Q(k )  and  R (k),  respectively.  We  assume  each 
target  has  a  separate  track  and  set  up  multiple  state 
equations.  Spurious  measurements  are  uniformly 
distributed  in  the  measurement  space.  Tracks  are  assumed 
initialized  at  an  initial  state  estimate  x(0),  contain  a  known 
number  of  targets  determined  from  the  scenario,  and  have 
associated  covariances  [i]. 

The  tracking  ID  filter  devotes  equal  attention  to  every 
validated  kinematic  or  ID  measurement  and  cycles 
through  measurements  until  a  believable  set  of  object  IDs 
is  refined  to  associate  one  object  per  track.  For  an  initial 
set  of  measurements,  a  hypothesized  number  of  tracks  and 
objects  of  interest  is  assumed  to  comprise  the  entire  set. 
Objects  are  possible  position  measurements  without  ID 
confirmation.  Successive  measurements  and  updates  from 
the  combined  feature  and  track  measurements  determine 
the  set  of  plausible  targets.  The  measurement  filter 
assumes  the  past  is  summarized  by  an  approximate 
sufficient  statistic  -  track  state  and  ID  state  estimates 
(approximate  conditional  mean)  and  covariances  for  each 
object. 

The  belief  measurement  information  Belk  =  M  •  Belk_i, 
derived  from  the  classification  measurements  of  the  target 
image,  represents  the  belief  update  states  of  the  ID 
measurements.  The  M  matrix  is  the  Markov  transition 
matrix,  which  represents  the  similarity  of  objects.  The 
similarity  of  objects  represents  how  the  belief  in  an  object 
type  may  be  related  to  other  objects  of  the  same  or 
different  type. 

The  measurement-to-track  association  probabilities 
are  computed  across  the  objects  and  these  probabilities  are 
computed  only  for  the  latest  set  of  measurements.  The 
conditional  probabilities  of  the  joint  track-ID  association 
events  pertaining  to  the  current  time  k  are  defined  as  0yotk, 

where  0/otk  is  the  event  that  object  center-of-gravity 

measurement  j  originated  from  object  o  and  track  t,j=  1, 
...,  mk;  o  =  0 ,  1,  ...,  On,  where  mk  is  the  total  number  of 
measurements  for  each  time  step  and  On  is  the  unknown 
number  of  objects.  Note,  for  purposes  of  tracking  and  ID, 
we  define  i  =  1,  ...  ,  mk  for  the  entire  measurement  set 
while  j  =  1,...,  mk  is  for  tracking  and  o  —  1...,  mk  is  for 
object  ID. 

A  validation  gate  for  each  object  bounds  the  believable 
joint  measurement  events,  but  not  in  the  evaluation  of 


their  probabilities.  The  plausible  validation  matrix :  Q  = 
|  coyt  |  is  generated  for  each  object  of  a  given  track  which 
comprises  binary  elements  that  indicate  if  measurement  j 
lies  in  the  validation  gate  of  track  t.  The  index  t  =  0 
represents  "the  empty  set  of  tracks"  and  the  corresponding 
column  of  Q  includes  all  measurements,  since  each 
measurement  could  have  originated  from  clutter,  false 
alarm,  or  true  object  [1]. 

For  a  track  event,  we  have: 


if  ejt  e  0;  [z]L  c=  t 
otherwise 


(3) 


where  measurement  [z]k  originated  from  track  t 


For  an  ID-belief  event,  which  is  above  a  predetermined  ID 
threshold, 


®oO(0)l  - 


A  1  if  0oO  e  9;  [Bel]ok  <=>  o 


(4) 


0  otherwise 

where  measurement  [Bel]ok  is  associated  (o)  with  object 


Since  MEIDA  is  tracking  multiple  objects,  o , 
assuming  one  for  each  track,  t ,  MEIDA  has  to  determine 
the  ID-belief  in  each  object  from  a  known  database 
comparison.  While  these  IDs  are  processed  over  time  to 
discern  the  object,  for  each  measurement,  MEIDA  must 
determine  if  the  track-ID  measurements  are  plausible. 
MEIDA  uses  the  current  ID-beliefs  to  update  the 
association  matrix.  If  the  belief  in  the  object  is  above  a 
threshold,  MEIDA  declares  the  measurement  /,  to  be 
plausible  for  the  target.  Note,  for  plausibility,  the 
threshold  is  lower  than  an  ID  declaration. 


5.2  Data  Association 

Since  we  have  assessed  the  continuous-kinematic 
information  and  the  discrete-classification  event,  we  can 
now  assess  the  intersection  of  kinematic  and  ID 
information  for  simultaneous  object  tracking  and  ID. 
Note,  ID  goes  beyond  object  detection,  recognition,  and 
classification,  where  we  define  ID  as  the  classification  of 
an  object-type  for  a  given  track  to  associate  an  object 
classification  to  a  track.  For  instance,  two  objects  of  the 
same  class  still  need  to  be  associated  with  a  specific  track. 
We  need  to  address  feasible  events  for  either  a  validated 
kinematic  measurement  or  a  validated  ID.  A  kinematic-ID 
joint  association  event  consists  of  the  values  in  Q 
corresponding  to  the  associations  in  Q/ot> 


Note,  we  define  the  indices  as  jot  since  O  is  the  number  of 
objects  which  is  equal  to  the  number  of  tracks. 

These  joint  events  will  be  assessed  with  “p”  weights  [1] 
to  determine  the  extent  of  belief  in  the  associations.  To 
process  the  believability  of  track  associations,  augmented 
with  the  ID  information,  we  set  up  a  matrix  formulation. 
For  example,  we  have  a  set  of  kinematic  measurements  z- 
with  a  Bel0  and  put  them  into  the  event  association  matrix 

as  illustrated  in  Figure  6.  The  upper  left  of  a  box 
represents  the  track  information  where  a  “1”  indicates  the 
kinematic  measurement  lies  within  a  gated  position 
measurement.  The  lower  right  represents  the  belief  in  an 
object  type  of  any  class  except  the  unknown  class  where  a 
believable  object  receives  a  “1”.  Columns  are  for  tracks 
and  rows  for  measurements.  These  generalized  equations 
propagate  ID-filtered,  predicted  ID  measurements  in  time. 
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Figure  6.  Tracking  and  Classification  Joint  Association 


MEIDA  processes  event  matrices  with  an  “AND” 
function  in  the  case  of  joint  association  allowing  for 
plausible  events  from  either  the  track  or  classification. 
[Note,  the  “OR”  function  could  be  used  for  high  clutter, 
but  was  not  used.]  To  determine  the  event  plausibility, 
MEIDA  uses  the  validation  region  for  track  measurements 
and  uses  a  threshold,  or  classification  gate ,  to  determine  a 
target-type  ID  match  associated  with  a  given  track.  Figure 
7  illustrates  the  “AND”  function.  Note,  MEIDA  rejects 
non-believable  measurements  and  measurements  that  lie 
outside  the  kinematic  validation  gate. 
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if0/ote  6* 

otherwise 


(5) 


where  (*)  measurement  [z]k  originated  from  track  t  with  a 
[Bel]ok  for  a  given  0ot  and 


cbiOt(0)  =  a>/t(0)  ©  a)o0(e). 


Figure  7.  Believable  Events  for  the  association  matrix. 

MEIDA  sets  up  the  state  and  probability  values  for 
the  determination  of  the  weights  assigned  to  these 
associations.  A  track-ID  association  event  has  [1] 

i)  a  single  object-type  measurement  from  a  source: 


(6) 


(12) 


t  %ot(it)=i  vy  (7) 

o  =  0 

ii)  and  at  most  one  object-type  measurement  ID 
originating  from  a  object  for  a  given  track: 

mk 

8/9)  =  I  %ot(0jot)  (8) 

7=1 

The  event  matrices,  Q  for  each  track,  corresponding  to  ID 
events  can  be  done  by  scanning  Q  and  picking  one  unit/row 
and  one  unit/column  for  the  estimated  set  of  tracks  except 
for  t  =  0.  In  the  case  that  MEIDA  has  generated  event 
matrices  for  an  estimated  number  of  tracks  with  different 
object  types,  MEIDA  needs  to  assess  the  combination  of 
feature  measurements  to  infer  the  correct  number  of  tracked 
objects  that  comprise  the  set.  The  binary  variable  S^(  0yotk) 

is  called  the  track  detection  indicator  [1]  since  it  indicates 
whether  a  measurement  is  associated  with  the  object  o  and 
track  t  in  event  0^,  i-e-  whether  it  has  been  detected. 

The  measurement  association  indicator 


-Z 

0 

MEIDA  decomposes  the  object-state  estimation  with  respect 
to  the  location  of  each  object  of  the  latest  set  of  validated 
belief-set  and  kinematic-set  measurements.  The 
measurements  have  been  used  to  get  the  classification 
beliefs  in  the  object  types,  to  set  up  a  simultaneous  tracking 
and  ID  recursion  for  each  object  in  the  set,  where  ID  is  the 
classification  of  each  object  for  a  given  track  of  data.  For 
each  object  measurement,  we  use  the  total  probability 
theorem  to  get  the  conditional  mean  of  the  state  at  time  k  can 
be  written  as: 


o 


A  t 

where  Xk|k  is  the  updated  state  conditioned  on  the  event  that 

the  ith  validated  object  measurement  is  correct  for  track  t.  The 
covariance  propagation  is: 


(9) 


indicates  measurement  j  is  associated  with  the  track  t  in 
event  0^. 


The  number  of false  measurements  in  event  6  is 


pk|k-i =  Fk-i  pk-i  (fLi)t  +  Qk-i> where  Q  k  = 

for  each  track  t. 

We  can  obtain  the  innovation  covariance  Sk  with  the 
associated  Rk  and  measured  Dk  by: 


Qk  o 

0  Bk 


m=  z  [i-  ye)i  (io) 

7-1 

The  joint  association  event  probabilities  are,  using  Bayes' 
Formula: 

p{0(/:)|z/c}  =  ?mmk)Mk)xk-'} 

=  |  0(k),m(k),ZkA ]  P(6(/c)  |  m (*)} 

m(k)  -  4>(k) 

=  -  nv{f#)M}ri  (io 

j=>  .  . 

where  c  is  the  normalization  constant. 

The  number  of  measurement-to-target  assignment  events 
0(£)  is  the  number  of  targets  to  which  a  measurement  is 
assigned  under  the  same  detection  event  [m(T)  -  <|)].  The 
target  indicators  8 1(0)  are  used  to  select  the  probabilities  of 
detecting  and  not  detecting  events  under  consideration. 

5.3  Fused  Track  and  ID  State  Estimation 

Assuming  the  targets  conditioned  on  the  past  observations 
are  mutually  independent ,  the  decoupled  state  estimation 
uses  the  marginal  association  probabilities ,  which  are 
found  from  the  joint  probabilities  by  summing  all  the  joint 
events  in  which  the  marginal  track  and  classification  events 
result.  The  beta  weights  [1]  are: 


Sk  =  Hk'Pk|k-l  (Hk) 


T  +  R  k,  where  Ru  = 


Rk  o 
0  Dk 


Since  Sk  is  the  innovation  covariance  update,  we  can  use  Sk 
to  gate  measurements  based  on  the  uncertainty  with  the 
associated  track  and  IDs. 


Validation:  At  k ,  two  measurements  are  available  for 

T  T 

object  o  for  a  given  track  t\  zk_1?  and  zk,  from  which 
position,  velocity,  pose,  and  ID  features  can  be  extracted 
form  the  belief  track  vectors.  Validation,  based  on  track 
and  ID  information,  is  performed  to  determine  which  track- 
belief  measurements  fall  into  the  kinematic  region  of 
interest.  Validation  can  be  described  as 


t  A/t  XT  rf^-.-l  ,  t  Alt  0  /■%  A  \ 

(Zk-Z k|k-l)  [Skl  (zk  -  zk|k-l)  -  y  for  /=  1  ...  7Mk  (14) 

where  y  is  a  validation  threshold  obtained  from  a  %2  table 
for  a  degree  of  freedom  of  14  (4  for  kinematic  states  and  10 
for  target  beliefs)  and  Sk  stands  for  the  largest  among  the 

predicted  track  belief  covariance,  i.e.,  det(Sk)  >  det(Sk)  for  t 
=  \2,...,n  where  n  is  the  number  of  states.  The  combined 
predicted  track  belief,  zk|k_i?  is  given  by 

E{zk |{7?}*  =  i,Zk-1}  where  s  is  the  set  of  object  beliefs  for 
a  track. 


Data  association  for  (3^  :  Data  association  performed  for 
each  belief  object- track  is  similar  to  that  in  PDA  and  the 
details  can  be  found  in  [1].  The  association  probabilities  for 
/  validated  object  measurements  are: 

t 

P/'  = - ej^,l=\,2,...,m°k  (15) 

mk 

b  +  £  e\ 

1=  l 

Po  = - V-  ,  (16) 

mk 

b  +  Z  e/ 

/=  1 

where  e|  =  Fq  7V(0,  )  (17) 

b  =m°k(l-PDPG)[PDPGVk]-1  (18) 


with  mk  defining  the  number  of  validated  object 
measurements,  PG  assessing  the  probability  that  augmented 
belief  track  measurements  fall  into  the  validation  region, 
and  PD  representing  a  detection  probability.  For  the 
MEIDA  case,  we  vary  the  innovation  covariance  (Sk),  PD , 
PG  proportionally  to  the  sensor  manager  collection 
resolution  (i.e.  higher  resolution  — »  higher  PD,  higher  PG ,  & 
lower  Sk).  The  lower  Sk  for  the  higher  resolution  is  a  result 
of  changing  the  prediction,  which  results  after  a  few  track 
instances.  The  volume  of  the  validation  gate  is 

rk  =  cd/2|sk|1/2,  (19) 


where  Cd  is  the  unit  hypersphere  volume  of  dimension  d,  the 
dimension  of  the  augmented  belief-track  measurement. 


Kinematic  belief-probabilistic  update:  The  object  belief- 
probabilistic  track  update  is  performed  as  a  full  rate  system 
to  combine  the  state,  innovation,  and  covariances. 


xk  =  xlnk.i  +  wk'i  p^v' 
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(20) 
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Wv 
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V  t 

where  Hk  is  the  measurement  matrix  that  is  calculated  for 
each  object  pose,  4>,  and  estimated  position  of  track  t. 


6  Initial  Results 


The  MEIDA  track  and  ID  method  is  evaluated  with  a 
Monte  Carlo  simulation  and  the  performance  metric  is 
position  state  error  (i.e.  RMS).  The  ID  information  is  a 
result  of  the  fused  EO/IR  discrete  result  where  the 
aggregated  comparison  between  true  image  versus 
corresponding  images  over  a  20  degree  window  for  the 
various  targets.  The  probability  results  from  the 
normalized  number  of  discrete  selections  over  the 
comparisons  (t  targets,  d  degrees).  The  high  resolution 
case  results  in  better  probabilities  than  the  low  resolution 
case.  As  detailed  in  the  Figure  8  below,  by  the  true 
trajectory;  the  targets  1)  start  with  position  and  velocity,  2) 
pass  by  each  other  at  a  close  distance,  and  3)  finish  with  a 
specified  direction.  There  was  added  noise  to  the  true 
target  position  and  clutter  comprised  of  5  spurious 
measurements  around  a  target.  Even  with  closely  spaced 
targets,  the  MEIDA  tracker  was  able  to  locate,  track,  and 
identify  the  targets  with  clutter. 


x  in4  Measurements  (without  clutter) 

1.05 - 

1.04 

1.03 

1.02 

1.01 


0.99 
0.98 
0.97 
0.96 

°'95  2000  2500  3000  3500  4000 

X 

Figure  8.  Tracking  scenario  with  noise. 
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Figure  9  shows  the  JPDAF  effectiveness  when  targets 
are  separated  by  a  large  distance.  The  separation  allows 
for  the  determination  of  a  validation  gate  size  that 
associates  the  correct  measurements  to  tracks.  However, 
as  targets  are  close,  the  tracker  combines  all  the  targets 
into  one  track,  due  to  the  clutter. 


Figure  9.  Tracking  without  ID  information. 

Figure  10  shows  the  same  case  as  Figure  9  with 
identification  information  that  helps  the  tracker  better 
associate  the  true  position  measurements  from  clutter. 


Target  beliefs  increased  throughout  the  run  as  the  target 
was  repeatedly  identified  as  the  same  target. 


Figure  10.  Tracking  with  ID  information. 

Note,  as  the  covariance  grows  in  target  position 
estimate,  the  sensor  manager  switches  from  a  low- 
resolution  mode  to  a  high-resolution  mode  to  better 
discern  the  target  types.  As  targets  separate,  the 
covariance  decreases  and  the  sensor  manager  switches 
back  to  the  low-resolution  mode  (show  in  Figure  11). 
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Figure  11.  Tracking  with  ID  covariance  information. 

7  Discussion  &  Conclusions 

Conventional  measurement  tracking  techniques  have 
difficulty  with  data  association  when  position 
measurements  are  close.  The  MEIDA  algorithm,  which 
uses  the  identification  information  from  an  EO/IR  sensor 
suite,  helps  to  associate  the  correct  measurement  to  the 
correct  targets.  In  the  presence  of  clutter,  the  novel 
algorithm  utilizes  parsimony  for  processing  by 
incorporation  of  covariance  information  to  select  sensor 
resolution.  The  MEIDA  can  be  utilized  in  a  time- 
constrained  weak-sensor  scenario  to  get  a  general  target 
location  and  a  positive  ID  (cued  high  resolution)  that  can 
be  used  to  augment  a  data  association  tracker. 

In  a  series  of  simulation  experiments,  the  MEIDA 
performed  well  resulting  in  a  desirable  solution  for  closely 
spaced  moving  targets,  and  at  a  faster  rate  than 
conventional  tracking  methodologies.  The  faster  rate 
resulted  from  a  reduction  in  the  gate  size  to  eliminate 
clutter.  The  presented  technique  demonstrates  promise  for 
multitarget  tracking  problems  and  warrants  further 


exploration  in  problems  where  environmental  effects, 
occlusions,  lost  sensor  data,  and  unknown  targets  require  a 
sensor  manager  to  control  sensor  resolution  to  optimize 
tracking  performance.  Future  work  will  explore  the 
sensitivity  of  the  results  to  a  higher  fidelity  EO/IR  target 
recognition  algorithm  over  more  operating  conditions  and 
utilize  methods  that  afford  track  initiation  (i.e.  IPDA  [8]). 
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